About | Support | Discussion Forum | Index | Service Status | ros @ Robotics Stack Exchange

Download object models and use them for object detection

Description: Download object models from the RoboEarth database and use them for object detection

Tutorial Level: ADVANCED

Next Tutorial: roboearth_stack/Tutorials/Record an object model

How to start the software

First, make sure the OpenNI camera driver is running:

roslaunch openni_launch openni.launch

Also, make sure that depth registration is enabled, see openni_launch#Quick_start for instructions on how to do that. Next, start the re_comm node (this can be skipped if you do not want to download object models from the RoboEarth database):

rosrun re_comm run

Now you can choose between two different detection services, re_kinect_object_detector or re_vision. Follow either one of the next subsections.

Using re_kinect_object_detector as object detection service

Start the object detection algorithm:

rosrun re_kinect_object_detector re_kinect

Finally, run re_object_detector_gui:

rosrun re_object_detector_gui detect

Using re_vision as object detection service

re_vision allows you (in contrast to re_kinect_object_detector) to use object models created with re_object_recorder with cameras that do not provide depth information. You need to provide it with some information about the camera though, as described on its ROS wiki page. You are also able to use re_vision with only the visual information of a Kinect camera. To do that, read below.

re_vision needs to read the camera calibration parameters from a /camera_info topic, which you can remap when starting the service. To start the detection service with any camera, run:

rosrun re_vision ObjectDetector /camera_info:=/your_camera_info_topic

You can also visualize the detection process by adding the -v flag when running re_vision. When this option is activated, the processed image is sent by the topic /re_vision/detector_visualization. You can visualize it with image_view:

rosrun re_vision ObjectDetector -v /camera_info:=/your_camera_info_topic
rosrun image_view  image_view image:=/re_vision/detector_visualization

Finally, run re_object_detector_gui, remapping the image topic to read raw images:

rosrun re_object_detector_gui detect /camera/rgb/image_color:=/image_raw

This process is valid for any monocular camera you use. There is also a launch file that makes this process easier if you are using a Kinect camera. In this case, to start up the detection service, just run:

roslaunch re_object_detector_gui re_vision_detector.launch

This starts the re_vision service and configures it to use the Kinect's RGB camera image. It also opens an instance of image_view to show a visualization of the process.

Adding local models

The user interface should open and look similar to this:

If you want to detect models you have already downloaded on you computer, click the Add Model Button. Now you should choose the directory containing the model files. Valid model directories should have a file called "meta.xml".

After you have selected a valid directory, click the OK button. If the selected directory contains a valid detection model, it will be added to the table in the user interface and sent to the object detection algorithm.

Downloading models from the RoboEarth database

To download a model from the RoboEarth database, click the Download button in the main window. A new dialog should open, looking similar to this:

Here you can query the RoboEarth database for objects by entering a prefix of the objects identifier. Try to enter "uploadtest", for example, and click the Search Button or press Enter.

If the query was successful, the table view below the search field should now contain some entries. These are the model files you can download. Note that there can be multiple object models for the same object. If you are using re_kinect_object_detector, the object model file you need is called kinectmodel.zip. Files called 2dcammodel.zip are for usage with re_vision. Select the model file of your choice by ticking the checkbox in the first column.

In the bottom of the dialog, you can see an input field containing your system's local directory (usually "/tmp"). This is where the models you select will be downloaded to. If you prefer a different location, you change it either by entering the path directly or clicking the "..." Button to open a directory selection dialog.

If you have selected all the object models you want, click the Download button. The program will now download and extract the object models to the location you specified, add these to the table view in the main window, and communicate these to the object detection algorithm.

Object detection

If the object detection algorithm detects one or more of the objects, it will open a new window showing the current camera image and marking the detected objects in red:

Also, a new tf transformation is published that specifies the objects estimated position.

Hint: If you have problems with the object detection the recorded model might not be dense enough. You could consider re-scanning the object and re-upload with more recorded point clouds from various directions.

If you forgot to start the object detection service, start it now and press Resend model list. This will send the list of selected models to the detection services again (this is also useful if the detection service crashed and you restarted it).

Viewing downloaded object models

To show the point cloud models you downloaded on the screen, open a new terminal and change to the location where you dropped the object models (usually /tmp if you did not change the path). This folder should contain the zipped object model (kinectmodel.zip) as well as the extracted one (directory $objectname_kinect).

To display the point cloud, run

rosrun pcl_visualization pcd_viewer $objectname_kinect/merged.pcd

(replace $objectname_kinect with the actual extracted object model folder name)

Pressing '5' will change the view to the colored point cloud. See pcl_visualization for more options.